1. Field of the Invention
This invention relates to a recording apparatus in which two or more different types of data, such as video and audio data, are multiplexed for recording on a recording medium, and more particularly relates to such a recording apparatus in which at least one of the types of data is encoded at a variable bit rate.
2. Description of Related Art
It has frequently been proposed to digitize video and audio signals and to record the digitized data on a recording medium such as an optical disk. Because of the very large quantity of data produced by digitizing such signals, it has also been proposed to perform compression coding on such data before recording. In this case, upon reproduction of the data from the recording medium, a decoding process is performed in order to reconstitute the digitized data. Where the reproduced data includes both video and audio data, such as constitute a television signal, the playing back of the video data must be synchronized with playback of the audio data. For this reason, it is customary to use a single apparatus for encoding and multiplexing the signal to be recorded. However, the multiplexing process does not necessarily have to be performed at the same time as the encoding process. During multiplexing, the different types of encoded data, such as the video and audio data, are read in at appropriate time intervals and at a proper rate. Then, after additional data, such as time stamps required for synchronous playback, is added to the video and audio data, a single data bit stream is produced.
In general, such data bit streams have a hierarchical structure. For example, the structure of a bit stream proposed by the Moving Picture Experts Group (MPEG) in International Standards Organization (ISO) Document No. 11172 is illustrated in FIG. 5. As shown in FIG. 5, the bit stream defined in ISO 11172 is made up of a sequence of packs, each of which includes a number of packets. Each pack starts with information such as a pack start code, a System Clock Reference (SCR) code and a system header. The SCR code is used for providing synchronization for the entire data storage and reproduction system including the encoder and decoder and is also used for controlling a buffer that will be described below.
Each packet begins with information such as a stream identification number, time stamps, and a required buffer size. The time stamps include a Presentation Time Stamp (PTS) and a Decoding Time Stamp (DTS). The PTS is indicative of a time at which the data was encoded by the encoder, whereas the DTS is indicative of a time at which the data was decoded by a decoder. The PTS and DTS are used to provide synchronous reproduction of different types of data. Compressed data such as video or audio data is inserted in the packet after the information which is located at the beginning of the packet. A packet which includes video data is called a video packet, and a packet which includes audio data is called an audio packet. Different types of data, such as video and audio data, are never mixed in a single packet.
According to the MPEG proposed standard described above, both the packs and the packets may be of variable length. The MPEG standard also proposes a bit-stream syntax which can accommodate simultaneously up to 32 substreams of audio data and up to 16 substreams of video data.
Details of the multiplexing process proposed by MPEG are described in 1-Annex A (Informative) associated with ISO 11172, and entitled "Description of the System Coding Layer", published on Mar. 27, 1992. The multiplexing processing standard proposed by MPEG includes some flexibility, as is shown by the fact that the packs and packets described therein can be of variable length. Accordingly, a variety of techniques can be employed for multiplexing in accordance with the MPEG standard. However, the most restrictive limitation placed on the multiplexing process by the MPEG standard is that the decoder buffer cannot be permitted to either overflow or underflow. For this purpose, the encoding and multiplexing apparatus includes a buffer which has the same capacity as the decoder buffer. The multiplexing process carried out in the encoding and multiplexing apparatus is controlled so that there is neither overflow nor underflow of the buffer provided therein. In the following description, "buffer" should be understood to mean the buffer memory provided in the encoding and multiplexing apparatus unless otherwise specified.
Other issues to be addressed in the multiplexing process include those relating to pack and packet lengths, pack configurations, and the ratio of different types of data packets to be inserted in each pack. It will be understood that "pack configuration" refers to the number of packets which make up each pack. It will also be recognized that the different types of data packets may include video packets and audio packets.
If input data such as video and audio data are all encoded at respective constant bit rates, the issues referred to above can be readily addressed. One way in which such issues can be resolved is illustrated by the example which is provided immediately below.
According to this example, video data is encoded at a constant rate of 150,000 bytes per second and the audio data is encoded at a constant rate of 24,000 bytes per second. Further, a constant packet length of 2,048 bytes is established for both video and audio packets. Since the video data rate is 6.25 (=150,000/24,000) times the audio data rate, and since the same packet length is provided for both video and audio packets, an insertion ratio of video packets to audio packets of 6.25 will permit the video and audio data to be reproduced synchronously. In other words, an audio packet must be inserted after every six or seven video packets.
Let it further be assumed that each pack is made up of three packets. In this case, a bit stream having the configuration shown in FIG. 6 is produced. It will be noted that, since the different types of data are all encoded at respective constant rates, an insertion ratio and a sequence for inserting the packets into the packs can be determined in advance on the basis of the data rates, the pack and packet lengths, and the pack configuration. Thus the multiplexing of the different types of data can be performed with relative ease. Furthermore, buffer control can be performed by monitoring the amount of data which has accumulated in the buffer. For example, if the buffer used for temporarily storing encoded video data is about to overflow, the quantization step used in quantization-encoding the video data can be increased, in a manner described below, to prevent the video data buffer from overflowing. In this way, buffer overflow is avoided because the amount of encoded data that is generated is temporarily reduced. On the other hand, if the video data buffer is about to underflow, pseudo data which is unrelated to the video and audio data can be inserted in order to temporarily increase the quantity of encoded data that is generated. In this way, buffer underflow can also be avoided.
It can be seen that a multiplexed bit stream can be generated with relative ease with respect to video and audio data generated at respective constant rates by means of multiplexing apparatus that operates at a predetermined ratio for inserting video and audio packets in a sequence of packs. Furthermore, when the video and audio data to be encoded happen to be such that highly efficient compression can be achieved, buffer underflow can be prevented by inserting pseudo data during the multiplexing process to assure that a constant bit rate is maintained. However, the insertion of pseudo data adversely effects the efficiency with which the recording capacity of a recording medium is used.
Furthermore, buffer overflow is prevented, again in order to maintain a constant data rate, by changing the value of a quantization step used in the encoding process. In particular, the quantization step is increased for portions of a picture which cannot be encoded with great efficiency or in portions having a relatively complex pattern for which compression cannot easily be performed. On the other hand, the quantization step is decreased for portions of a picture in which efficient encoding can be performed or in portions having a relatively even pattern for which compression is easily performed. As a result, uniform picture quality for the entire image cannot be assured. However, to achieve uniform picture quality, it can be contemplated to encode video data at a variable rate. Specifically, for portions of an image that cannot be efficiently encoded, the encoding rate can be temporarily increased to provide more code bits for the encoding process. On the other hand, for portions of an image in which highly efficient encoding can be performed, the encoding rate can be temporarily decreased. Thus, by adaptively controlling the encoding rate as described above, uniform picture quality across an entire video image can be obtained. Because the data encoding rate provided in this case is variable, it is not necessary to insert pseudo data, so that the recording capacity of the recording medium can be used efficiently.
Reference is made to the disclosure of multiplexing and demultiplexing apparatus used for processing variable-rate-encoded data in U.K. Patent Application GB 2 259 229, published on Mar. 3, 1993. However, a detailed description of techniques for controlling a code buffer employed in such multiplexing apparatus is not available.
Incidently, in the variable-rate-encoding process described above, the encoding bit rate varies adaptively, and so cannot be predicted. Therefore, the ratio at which different types of packets, such as video and audio packets, are to be inserted in packs during the multiplexing process cannot be determined in advance. Further, it is more difficult to carry out multiplexing with respect to variable-rate-encoded data than is the case with respect to constant-rate-encoded data.